As complex converged telecommunication networks designed to provide a wide range of services to customers become more prevalent, the need for thorough testing becomes important prior to operationally releasing any replacement module or improvement to any part of the system. For example, operational support systems (OSSs) in telecommunication systems have many different elements that require frequent updates and improvements when new services are introduced to customers. Such OSS systems typically include order management, design and engineering, carrier interconnection, service provisioning, inventory, service activation, billing, customer care and service assurance modules. These modules are able to interact with each other and various modules and networks and also must be able to accept input from and provide information to external elements, such as sales, engineering and technician personnel. Accordingly, such systems are complex and require careful configuration control. When a new element of a system, such as a billing module, is inserted into the system, an integration system test is typically conducted to verify the configuration, compatibility and proper operation of the module. In the case of today's large converged telecommunication networks, such tests can be correspondingly very large and complex in scope.
Typically, such tests involve creating a comprehensive set of test data and running that data through the new module/system element under a variety of test scenarios involving various system configurations and loads. These test scenarios are typically created independently of the system under test and involve integrating special test equipment with the system prior to testing. As the data is processed by the system, intermediate results are monitored at various points throughout the system in order to ensure proper functioning of the system. Any faults are detected through the detection of anomalous data at these monitoring points. As a result of the complexity involved in integrating test equipment and validating test data/scenarios, such large integration systems tests (ISTs) are typically long in duration, expensive and risky.